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ABSTBACT / 

Naval researchers studied the effects of delay in the 
presentatia of visual information on pflot performance. Simulated 
carrier landing tasks were performed by subjects using a visual 
display ^generated by a computer. In one part of the experiment pilots 
were a^Jced to "fly" carrier approaches with and without a 0.1 second 
dela^/ in the visual scene presented to them. In the second part of 
the^ experiment, pilots were asked to "fly" carrier approaches several 
t^es during which six pilot control inputs were recorded, 
/^atistical analysis of the data generated by the experiment 
indicated that the differences between, mean performance with delay 
and with no delay were not significant. Variances of lateral control 
deflection and force were signif icantly- different for the delayed 

esentation and the nondelayed presentation, but the analysis of the 
other four control inputs did not reveal statistically significant 
differences. (CH) 




4e j(( ;|( 4e ?|c 4e 4e 4e 4e 4e 4e :9c 4e 4e 4e 4e 4e 4e 4e ♦ 4t ^ 4e ♦ 

* Documen.ts acquired by ERIC include many informal unpublished •* 

* materials not available from other sources. EEIC makes' every effort * 

* tQ:r^btain the best copy available. Nevertheless, items of marginal *' 
-^♦^reproducibility are often encountered and this affects the quality * 
.♦of the microfiche and hardcopy reproductions EHIC makes available * 

* via the ERIC Document Reproduction Service (^JDRS) . EDRS is not * 

* responsible for the quality of the original document. Reproductions * 

* supplied by EDBS are the best that can be made from the original. * 

4e atesfesfe 3^ 4e4t3|t3|t4e«« ♦♦♦♦♦♦♦♦♦♦♦♦♦♦ ♦♦♦♦^eafeafeafeafeatc^t^t^^^^ 




m 



TECHNICAL REPORT: NAVTRAEQUIPCEN IH-250 



THE EFFECT OF DELAY IN THE PRESENTATION OF VISUAL 
INFORMATION ON PJLOT PERFORMANCE 



Analysis and Design Branch, 
Computer Laboratory, and 
Human Factors Laboratories 
Naval Training Equipment Center 
Orlando, Florida 32813 



Final Report for Period April 1974 - July 1975 



15 December 1975 




us DEPARTMENT OF HEALTH 
EDUCATION & WELFARE 
NATIONAL INSTITUTE OF 
EDUCATION 

□ ."EO EX&C^u^ PE'CE'-'ED p POM 

A-f NO * PO N'^SOf ^ EA OR OPINIONS 
S'^A'ED DO MECESSA© RfPRE 

SEN' Of f C NATIONAL ■ Ni S 'J 'JJ-X e"QV 
ED^CAT ON POSi"^ ON JiJ5 ^rTl'^Cv 



DoD Distribution Statement 



,^^pproved for public release; 
distribution unlimited. 



NAVAL TRAINING EQUIPMENT CENTEPx 
ORLANDO, FLORIDA 32813 




TECHNICAL REPORT NAVTRAEQUIPCEN IH-250 



THE EFFECT OF DELAY IN THE PRESENTATION OF VISUAL INFORMATION ON PIWT 
PERFORMANCE 



Fred R. Cooper 

Analysis and ^j^sign Branch 

William T. Harris 
Computer Laboratory 

Vincent J. Sharkey 
Human Factors Laboratory 



December 1975 



GOVERNMENT RIGHTS IN DATA StAtE^ffiNT 

Reproduction of this publication in whole or In 
part is permitted for any purpose of the United 
States Government. 



Approved: 
GLENN W. HOHMAN 

Head, Analysis and Design Branch 



DR. L. D. HEALY 

Acting Head, Computer Laboratory 

JAMES S^UVA \ 
He^drj^uman Factors Laboj'atory 



HUGH HALPJN, gDR, USN 
Deputy Director 

Research and 'Technology Department 



NAVAL TRAINING EQUIPMENT CENTER 
ORLANDO, FLORIDA 
32813 



UNCLASSIFIED 



ERIC 



SECuSlTY CLASSIFICATION OF THIS PAGE rW7i»n Data Bnfrad) 



V REPORT DOCUMENTATION PAGE 


READ INSTRUCTIONS 
BEFORE COMPLETING FORM 


1. REPORT-NUMBER ^ 2. GOVT ACCESSION NO, 

NAVTRAEQUIPCEN IH-250 


3. RECIPIENT'S CATALOG NUMBER 


4, TITLE (mnd Subttila) 

The Effect of Delay in the Presentation of 
Visual Information on Pilot Performance 


5. TYPE OF REPOPT 4 PERIOD COVERED 

Final Report 

Apr 1974 - July 1975 


6. PERFORMING ORG. REPORT NUMBER 

NAVTRAEQUIPCEN IH-250 


7. AUTHORf*; 

Fred R, Cooper 
William Harris 
V incen t j • onarKey 


8. CONTRACT OR GRANT NUMBERr«) 


9 PERFORMING ORGANIZATION NAME AND ADDRESS 

Analysis and Design Branch N-2211 

Computer Laboratory N-214 

Human Factors Laboratory N-215 

Naval Training Equipment Center. Orlando. PL 


10. PROGRAM ELEMENT. PROJECT. TASK 
AREA A WORK UNIT NUMBERS 

NAVTRAEQUIPCEN 
Task No. 6948 


n. CONTROLLING OFFICE NAME AND ADDRESS 

Department of the Navy 

Naval Training Equipment Center 

Orlando, Florida 32813 


12. REPORT DATE 

December 1975 


13. NUMBER OF PAGES 

77 


t4 MONITORING AGEh^CY NAME 6 ADD RESS( 1 1 dUtar^nt irom ConiroUIng Ottlea) 


15. SECURITY CLASS, (ot thla rapottj 

Unclassified 


15a. DECLASSIFICATION/ DOWN GRADING" 
SCHEDOLE 


16. DISTRIBUTION STATEMENT fonft/» /?«porO 

Approved for public release; distribution unlimited 


17. DISTRIBUTION STATEMENT (o/ tA« mbairmct 9ntarad (n Block 30, It dUtattit from^Raport) / 

M ♦ 


18. SUPPLEMENTARY NOTES 

/ 

1 


19. KEY WORDS (Coniinua on tavtaa aida It nac^aaarf afd Idantity tsy btock nximbaO 

Flight Simulators Huma|i Response \ 
Visual Systems 
Delay Effects 

Training \ 


20. ABSTRACT (Contlnua on rawaa aida K nacatamry and Idantity by block mimbar) 

The effects of delay in the presentation of visual information on pilot per- 
formance during "simulated carneif: landing tasks were investigated! The TRADEC 
research flight simulator was used in conjunction with an Evans and Sutherland 
IDS 1 calligraphic visual display system for several differerit initial condi- 
tions, with and without delated visual presentation, in conducting evaluations 
of pilot learning performanc^ an^ piloting technique. The experimental con- 
struction, conduction, data ^^^alysis and results are presented herein, 

i 



DD 



1 JAN 73 



1473 



S/N 0102-0^4-6601 I 



UNCLAS^iED 



SeCUNITY CUASSIFICATION OF THIS PAOC (Whan DmU KntaraO 



.1. 



NAVTRAEQUIPCEN IH-2S0 
SUMMARY 

A considerable effort has been undertaken at' the Naval- Training Equip- 
ment Center's Analysis and Design Branch, Computer and Human Factors Labora- 
tories toward answering three questions regarding computer generated visual 
system technology. The questions were: / 

a. Do visual system presentation delays on the order of 0,1 seconds 
have any adverse effects on pilot trainee learning ability? . . 

b. Do the presentation delays cau5e the pilot subjects to exercise 
their piloting skills differently than when their visual stimuli are not 
delayed? ^ 

c. What is the nature of the differences in piloting techniques 
utilized when the pilot's visual stimuli have been delayed, if any? 

Questions a, b, and c have been answered by a two experiment study 
for the specific task of landing an aircraft simulator, with performance 
similar to an F-4, on an aircraft carrier visual display generated by 
computer generated imagery. 

Experiment 1 of the study addressed the^irst question posed. Twelve 
pilot subjects of varying age an4 background were asked to "fly" carrier 
approaches both with and without a 0,1 second delay in the visual scene 
presented to them. The performance criterion of merit was the number of 
trials required for the subjects to complete three successive carrier 
arrestments. 

Experiment 2 of the study addressed the second and third questions 
posed. For Parti of Experiment 2, twelve pilot subjects were asked to 
"fly" carrier approaches until five successful carrier arrestments were 
made. Real time data recording was used to record six pilot control 
inputs, A statistical unit of measure known as the variance was computed 
for each of the* control inputs. These variances were compared for the 
delay and lio-delay cases using some standard statistical analytical 
procedures known' as multivariate analyses, 
/ 

Part 2 of Experiment 2 addressed question c and utilized the data 
gathered under Part 1. Fast Fourier transforms were performed on the 
pilor control inputs for the delay and no-delay conditions transforming 
the ^eemingly random time histories to the frequency domain for easier 
int^erjiretation. The frequency spectra fot the delayed environment of 
thy^ ^corded control parameters were compared to those for the non-delayed 
eiivironment . 

/ /' / 
The results of this 



^ ^. In Experiment 1, the difference between the mean number of trials 



:s study indicated: 
^ . 1, the difference 

^ required by the piloj/t subjects to reach criterion performance in the delay 
condition and the mean number in the no-delay condition was not statistically 
significant. In fa<it, except for the earliest trials, the differences 
between mean performance with no delay and mean performance with delay 
were practically non-existent. 
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SECTION I 
INTRODUCTION 



Pecause^of the current national economy, the fuel shortage, concern 
for ecology, and the ever increasing complexity and cost of modem weapon 
systems, there is, and will lilj:ely cbntinue to be, emphasis on the develop- 
ment and utilization of sophisticated ^f light simulators. Military and 
commercial aircraft users are investing heavily in flight simulators equipped 
with visual systems and in visual systems to be attached to existing flight 
simulators. 

In general, visual simulators are conceiv^ as add-on systems/to flight 
trainers. Investigation of interfacing such systems has been, hi/storically, 
and typically, less thaa rigorous. Addition of one system to aviothex seems 
inevitably to affect the operation of the combination. Such \s the case 
with visual systems when attached to flight simulators. 

An inherent delay exists between the time a visual system receives 
its inputs and th'e time a visual presentation is displayed. For example, 
the Computer, Generated Image Advanced Development Model visual system 
attached to Device 2F90, a TAr4J OFT, at Kirigsville Naval-.Air Station (NAS), 
T«xas., in late 1973, required a little in excess of 100 ms to^enerate a 
visual scene. This time delay added to the 50 ms update cycle time of the 
2F90, represented . I 200 percent change in time related effects on the pilot's 
control responses. 

The question naturally axose as to what effect this additional delay 
is likely to have on the training effectiveness of a flight simulator ^ 
system. 





\ 

^Healy, L. D. and Cooper, f\ R., "Verification of Simulator Performance by 
Frequency Response Measurement, Proceedings of the 6th NAVTRAEQUIPCEN/ 
Industry Conference, Nov 13-15, 1973, NAVTRAEQUIPCEN IH-226. 

2 

O^Connor, F. E., CAPT USN, Dr. B. J. Schinn, and Dr. W. M. Bunker, ^Trospects, 
Problems, and Performance: A Case Study of the First Pilot Trainer Using 
CGI Visuals, Proceedings of the 6th NAVTRAEQUIPCEN/Industry Conference, 
Nov 13-15, 1973, NAVTRAEQUlPCfiN IH-226. 
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SECTION II 
STATEMENT OF THE PROBLEM 

The purpose of the experiment was to attempt to answer the following 
questions: 

a. Does a aOO delay of a visual presentation affect pilot learning 
performance? / 

b. Do pilots perform their piloting skills differently when their 
visual stimuli have beeiKdelayed for 100 ms? 

c. .. If pilots do perfortiktheir skills differently when visual stimuli 
are delayed 100 ms, in what wa>{s) is their performance different? 
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- " section iii y 
experiment' description 



APPROACH ^ 

A- ■ 

^The previous ques^tions were addressed by two experiments. Experiment 1 
was deigned to answer Question a. Experiment 2 was designed to answer 
Questions b and c. • \ . ^ 

\ • ... ■ •• > - \ 

The approach taken^ ta answer Question a was to design specific carrier 
^jproac!! tasks which incorporated botTi delay and no-del^y conditions to be 
learned by pilot subjects. The pilot subjects were then required to fly 
the tasks. An analysis of the .number of carrier approach trials taken to 
achieve an established successful criterion of^^e^formance was then con^cted. 

The approach taken to answer Questions b and.c was to focus the imesti- 
gation on the pilot/simiil^tor interface — the flight controls. Pilot Control 
displacements and forces WM'e measjured while flying specific carrier approach 
tasks with and without 100 milliseconds (ms) delay. An analysis o^j^ihe re- 
corded measurements was accon^lished to determine if pilots manipulated the 
controls with more or less disp]bacements and/or with more or less forces 
when their visual stimuli were delayed. Finally, the^ measurements of control 
displacements and forces were subject^ed to a Fourier analysis to exa^ne, in 
the frequency domain, the effects of the lOO ms visual presentation ' delay on 
flight control activity. \ «i 

HARDWARE AND SIMULATION SOFTWARE USED 

The experiments were conducted with the Naval Training Equipment Center's 
(NAVTRAEQUIPCEN » s) TRADEC F-4 Flight Simulator. This simulator system con- 
sists of a Xerox Data System Sigma 7 digital Computfe^ with a full complement^ 
of general purpose digital computer peripheral equipment (figures 1 and 2), 
a four-degree-of-frejedom mot ion platform (figure 3), a variable configuration 
simulated aircraft cdckpit (figure 4)', and an operator 's^control console 
(figure 5). . - ^ ^ \ 

The computer system hardware consists of 48,000 words o£>,core storage, 
13.7 million bytes of random access (disc memory, four magnetic tape drives, 
a high-speed line printer, card reader, card punch, paper tape reader/punchy 
and a Calcomp incremental plotter. The sin^ulator software is a program 
which simulates the F-4 airblraft. The F-4 simulator is utilized in the con- 
duct of research in various aspects of simulation techniques and of human 
factors relating to simulation. The program is. written to support operator's 
console functions such as establishing modes of flight, recording of data, 
aiding in conducting tests and establishing different conditions and confi- 
gurations of flight. The program allows recording of up to 165 selectable 
parameters on magnetic tape each program iteration cycle, i.e., every 50 ms. 

The simulation program was modified to provide appropriate operator 
control of the conduct of this experiment. Program modifications provided 
for: 

- 13 _ 
11 
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NAVTRAEQUIPCEN IH-2S0 

a. Operator insertion of test subject identification and carrier 
approach task conditions (e.g.. Delay /No-Delay, Task Selection, etc.). 

b. Presetting the position of the simulated aircraft to one of three 
selectable points in space from which carrier approaches began. 

/ / " , ' 

c. Operat;or release of control of the simulator to a pilot subject 

enabling him to fly an approach and attempted arrestment on a visually 
depicted carrier. ' ^ 

d. Appropriate termination of each carrier approach, 

e. Control of recording selected data on magnetic tape during each 
approach, - ' 

The F-4 'simulator was interfaced with an Evans and Sutherland Line 
Drawing System (LDS) I line drawing visual CRT display system which provides 
a 19 horizontal by 19° vertical field of view. This monochromatic visual 
system consists of a line drawing scope s,hown in figure 5, a special purpose 
high-speed processor, figure 6, and an associated slave scope located in the 
simulated cockpit in view of the pilot, shown in figure 4, The special pur- 
pose high-speed processor accepts aircraft and aircraft carrier position and 
orientation information from the simulator computer and produces the correct 
perspective picture at the two. display stations in real time. The time re- 
quired for the visual system to compute and display the aircraft carrier 
scene used in these experiments varies from 12.5 ms to 25 ms. The time 
.taken within this range depends upon the number of lines that are in view 
of the pilot's eyepoint, which is dependent upon the distance between the 
aircraft and the aircraft carrier as the approach to arrestment progresses. 

The F-4 simulator program's iteration cycle is 50 ms. Position and 
orientation of the aircraft and aircraft carrier are computed each program 
iteration. The method of simulating 100 ms additional delay in the visual 
system was accomplished by withholding, from the Evans and Sutherland visual 
system, this airpafaft and carrier positional information for two program 
iteration cycles (2 iterations x 50 ms per iteration = 100 ms). This was 
accomplished by softwatre, the implementation of which is illustrated in 
figure 7, Carrier and aircraft positioning infQrmation was stored in buffers, 
the first buffer containing the position information calculated during the 
preceding program iteration cycle, (therefore 50 ms old), the sercond buffer 
the iteration cycle before that (100 ms old), etc, with the 9th buffer hold- 
ing the information calculated during the 9th previous iteration (i.e,, 450 
ms old). 
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. Sutherland, Ivan E. and Dan Cohen, "Display Techniques for Simulation," 
Technical Report: NAVTRADEVCEN 70-C-0025~1. 
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SIGl-lA 7 COMPUTER 
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Figure 7. Time Delay Implementation and 
Signal Interfacing i 
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At the end of each simulation^ program iteration cycle the contents of 
buffers 0 through 9 werje^ transferred, or shifted to the adjacent buffer. 
Information in the 9th^ buffer was discarded. Selection of a given buffer 
ta be presented to the Evans and Sutherland visual system therefore deter- 
min^d the amount of visual system time delay simulated. The subject experi- 
ment utilized the selection of "buffer" 2 when a delayed task was to be flown 
and "buffer" 0 when a no-delay task was to be fl^. 

The implementation just described resulted in effectively adding 100 ms 
time delay to the actual time required by the Evans and Sutherland system to 
produce and display the position of the aircraft carrier scene. Therefor^, 
the actual visual cue delays presented to the pilot subjects was 12.5 ms to 
25 ms for the no-delay condition and 112.5 ms to 12S ms for the delayed con- 
dition. 

PILOT SUBJECTS USED 

Sixteen Navy, Marine and Air Force pilots and former pilots, assigned 
to or employed as civilians, by the NAVTRAEQUIPCEN, or employed and self- 
employed in industry in the Orlando area, volunteered their time to serve 
as pilot subjects in 4:he experimentation. (Table 1 contains a summary of 
their flying experience.) All but two were carrier qualified from two and 
one-half years to twenty- five years ago. 

TASifS PERFORMED BY PILOT SUBJECTS 

The tasks selected were rather exacting and purposely so, for it was 
thought that if an artificial de:^y of 100 ms were to have an effect, it 
would show up more readily in the. more diffiqult parts of the flight training 
regimen. 

The basic task for the pilot subject was to learn to land a simulated 
aircraft on the carrier deck displayed on a Cathode Ray Tube (CRT) screen. 
Six variations of this basic task were used. Two were considered a priori 
to be of least difficulty, comparatively speaking, two of moderate""difficulty 
and two, the most difficult. This was done in order to afford the pilot sub- 
jects some early opportunity of success to prevent possible discouragement on 
their part and also in the later analysis to determine if an interaction exis- 
ted between Delay and Task Difficulty. 

Certain initial conditions. were common to all six task variations. In 
each case, the carrier moved at a rate of thirty-five (35) knots. The air- 
craft was always positioned one (1) nautical mile from the carrier at an 
altitude o£ three hundred ninety (390) feet and at an airspeed of one hundred 
thirty-five (135) knots (i.e., on the glide slope and at the correct airspeed), 
Except for pilot control positions, initial conditions were the s^e for 
each approach trial. Each successful approach trial required about 30 
seconds flight from the time the pilot subject was given 96ntrol until 
approach termination occ\ia::^ed. 

The six task variations we^^^^ follows: 

Task A (Least Difficult) 
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J 

The aircraft was set 600 feet to the ri^t of the center line of the, 
carrier's angle deck, figure 8; The pilot subject was required to make a 
left tym to line up on the center line of the carrier's angle deck. 

- Task B (Le^st Difficult) 

Thei aircraft was set directly on the glide' slope and on the center 
line of the carrier's angle deck, figure 9. No tums^were required and 
the^ilot subject's objective was to hold the aircraft on the glide slope 
until arrestment, 

.Task C (Moderately Difficult) 

This task was the same as Task B, figure 9, except that an arbitrary 
''level of turbulence representative of "light turbulence" flying conditions 
was added to the simulator motion system. ~ \* 

Task D (Moderately Difficult) 

The aircraft was set 600 feet to the left of the center line of the 
carrier's angle deck, figure 10. The pilot subject was required to make 
W right turn to line up on the angle deck's center line. , / 

/' 

Task E (Most Difficult) ^/ 

This was the same as Task D, figure 10, (right turn /^quired from 
600 feet to the left of the angle deck center line) wit^'an arbitrarily / 
selected more severe Aevel of turbulence, representative of "heavy tur- 
bulence" flying conditions, added to the simulator mbtion system. 

Task F (Most Difficult) 

This was the same as Task A, figure 8, (left turn required from 6QD 
feet to the right of the angle deck center line) with the more severe leA7el 
of turbulence added to the problem. 
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There were five conditions which had to be met in order for a trap 
(aircraft arrestment) to be successful: 

a. The trap area on the carrier deck was rectangular in shape and 
simulated a carrier deck area 50 feet wide by 80 feet long. A trap was 
possible if the aircraft center of gravity was in an altitude range of . 
64 to 69 feet above sea level and within the trap area. 

hi The landing gear had to be down. 

c. The rate of descent of the aircraft had to be less than or equal 
to 1000 feet per mifiute^as it entered the space defined in paragraph a. 
above.* ^ 

d. Th^ aircraft could not be pitched down more than two degrees from 
horizontal aiid not be pitched up more than eighteen degrees from horizontal 
as it entered the space defined in paragraph a. 

24 . 
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Figure 10. Visual Display Starting Position (Right) 
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The aircraft could not he rolled to the left or right more than 
fifteen degrees from horizontal as it-^fttered the space defined in para- 
graph a. y 

DESCRIPTION OF EXPERIMEliTS " * • 

Two experiments were designed to answer ^the questions stated in Section 
II, Statement of the Problem* The first experiment was designed to address 
Question a of Section II* The second experiment was designed to address 
Questions b and c of Section II. 

The procedure common to both experiments was as follows. Each pilot 
subject was brie^^^ before entering the simulator cockpit. After the 
briefing and while the pilot subject was buckling into the cockpit seat, 
the operator entered the pilot's identification Code, task selection, and 
delay/no^delay control code into the simulator pr9gram. The operator then 
preset the simulated aircraft *s position to a point in space associated 
with the selected task. The pilot at ±his time could see a visual display 
of an aircraft carrier as s.een from 390 feet altitude, at ^ distance of 
one mile, and either 600 feet left of, 600 feet right of^ or directly 
aligned with the center line of the carrier's angle deck, figures 10, 9, 
and 8 respectively. The simulated flight airspeed was set at 135 knots. 
When the pilot subject indicated he was ready, the operator released con- 
trol of the simulator to the pilot. The pilot was then completely in 
control jf^the flight simulator. Recording of the pilot^s flight contt'ol 
activity ^g^magnetic tape began at the instant the operator released control 
to the pilot. The pilbt^.was then required to fly the approach visually to 
the displayed carrier and attempt an arrestment. Automatic data recording 
every SO ms oii^agnetic tape continued until. the approach terminated with 
an arrestment, as^bolter, a wave off, or a crash. Upon conclusion of the 
approach, the oper^or reset starting conditions as described previously 
so that the pilot could attempt another approach. The .pilot subject con- 
tinued making approaches in t:his fashion until successfully completing the 
establisl^ed success criterion for the experiment. After successfully 
completing, a task, the operator inserted appropriate task selection and 
delay cbdes into the""]prDgrajn to set up the subsequent task. ^ 

EXPERIMENT 1 

PROCEDURE. Twelv^ of the pilot subjects practiced each task until each 
was proficient in task performance. A pilot subject was considered to 
have learned a task if he made three successful arrestments in a row in 
that task. The dependent variable was number of trials to criterion pei^- 
formance for each task. 

The tasks were always presented in the a priori order of difficulty, 
that is. Tasks A and B preceded Tasks C and D and the latter preceded Ta^sks 
E and F. Within this general order, however, the Delay ys No-Delay condi- . 
^tion was interleaved so that one condition may not have an obvious advantage 
over the other due to "practice effects." The order in which the pilot 
subjects leimed the tasks is summarized in table 2. Each pilot subject 
was assign^ to a presentation order at random with the restriction that 
the last pilot subjects were assigned to orders to maintain the^verall 

. : ". 25 2 7 
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balance in table 2, The^^^humbers in the feody of table 2 specify the order 
in which each pilot subject learned the tasKs under the two (Delay/No-Delay) 
conditions. ^ ' ^^.^ 

Table 2 indicates that six pilot subjects learned Task B first, three^ 
in the Delay condition and three in the No-Delay condition. Those three . 
that had learned Task B in the No Delay condition then learned Task A in 
the Delay condition. Those three that had learned Task B in the Delay 
condition then learned Task A in the No-Delay condition. The other six 
pilot subjects learned Task A fi^st, three with No-Delay and three with 
Delay, and then learned Task B ^second with the conditions reversed. Tasks 
C and D, and then E and F were learned in the orders indicated in table 2, 
The pilot subjects were not informed of the Delay or No-Delay conditions. 

Overall then, each of the twelve pilot subjects learned six tasks, 
two tasks at each of the three Difficulty levels, and at each Difficulty 
level, one under the No-Delay condition and one under the Delay condition. 
Each pilot subject was considered to have learned each task when he per- 
formed three successful entrapments in a row (successful performance). 
The dependent variable was the number of trials on each task required tt) 
reach successful performance. 

DATA RECORDED, A log was kept of each pilot subject's carrier approach 
trials for each task and each delay condition. The log contained the 
results of each approach, i.e., trap, bolter, wave off, or crash. Figure 
11 is a sample of the log. ^ 

The date, pilot identification code, and task sequence designation 
were recorded on each page of a subject's record. The approach trial 
number, the approach outcome (e.g., wave off, bolter, crash, or trap), 
the number of wire caught (wire 1 through 4), the task designation (taskaf^ 
A through F, with indication of delay or no-^delay), and remarks, as / 
applicable, were recorded for each approach trial. The remarks column was 
.intended primarily to ilote spont^eous, off-hand comments ftom th^ subject 
pilot that may have supported, or been relevant to, the analysis of the V 
experiments 

As indicated in figure 11, a ta^k was flown until the subject 
achieved three successive traps. The next task called for in the given 
pilot's task sequence was then set up. The subject continued in this 
fashion until completing_all six tasks, 

DATA ANALYSIS AND RESULTS. The analysis of variance model used in the, 
data analysis is a special case of three-way classification mixed model 
in which the Delay/No-Delay condition and the Task conditions are fixed ^ 
constants and tlje assignment of the pilot subjects was a random variable. 



n — ^^.^ 

See McNamar, Quinn: Psychological Statistics, John Wiley § Sons,"^Y#, N,Y. 
1969, pp 364--S71 ^ X 
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Figure 11. Experiment Part 1, Sample Log 
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The results of rh^ analysis of variance are presented in table 3. The 
main interest was testing for effects of the two manipulated variables (Delay/ 
No-Delay and Task Difficulty) and their interaction. For the influence of the 
Delay/No-Delay condition on pilot subject performance, F = 0.531 which is ob- 
ylously not significant. For the effect of Task Difficulty, F f 6:666 which 
is ,a statistically significant ratio (.0^>p>.01, df = 2. 22) . ^ The Delay by 



TasTc iitteraction, F = 0.89, is also not a significant result, 
tests ar^ avail^^le in this model. 



No further 



The f^t that task condition has a significant effect on pilot subjects' 
learning performance is not surprising. Recall that the ^askslwere presented 
roughly in the order of difficulty that was agreed upoiT^s^^^gri^. Thus, <two 
factors influenced the pilot subjects' learning performance frpni task to task 
throughout the^xperiment. The first factor (task difficilljy7 presented in 
the order - relatively easy to difficult) tended to cause a greater number 
of trial s-to-crirerjon to be required for the more difficult task. The 
second factor, practice effect, operated in the opposite direction and tended 
to cause fewer triais-to-criterion as time went by after longer practice. The 
effect of the first Vactor, difficulty (perhaps because the range was narrow), 
was overshadowed by the effect of the second factor, practice, and the gen- 
eral diminution of they trials-to-criterion on the latter tasks is evidenced 
by the significance of\the Task factor in the analysis of variance. 

Further evidence oii this point is presented in table 4. Each average 
in table 4 is based on tne performance of twelve pilots. The diminution 
of the average number of Vrials-to-criterion is especially noticed in pro- 
gression from the least difficult to the moderately difficult tasKs. Per- 
formance' levels, off therea^er so that the^"most difficult" tasks were 
learned in approximately the^same number of trials as were the "moderately 
difficult." 



Within each "Difficulty" level, however, the erences between 
the Delay aftd the No-Delay conditions are of no statistical nor practical 
significance. The only possible exception from the practical point of 
view lies in the "Least Difficult" task level where the average number of 
trials-to-criterion was greater under the Delay conditions. This difference 
was due solely to the performance of one pilot who took 157 trials-to- 
criterion in the Delay condition (his first task) and then made only one 
subsequent error during the remainder of the experiment. 
> 

The^nain conclusion from this part of the study is that, overall, the 
introdii^ion of a 100 ms delay in presentation of the visual information 
had no effect on the learning by the pilQt subjects. 



^For \he choice of the error terms in these tests see McNamar, Quinn, 

ibid., pp m-zny 
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Table 3. ANALYSIS OF .VARIANCE FO^' PERFORMANCE 

"SCORES FOR 12 PILOT-SUBJECTS FOR TWO "DELAY" 
CONDITIONS AND THREE LEVi^LS OF TASK DIFFICULTY 





SUM 


DEGREES 
















SOURCE 


SQUARES 


FREEDOM 


ESTIMATE 


F RATIO 


Delay (D) 


234.72 


1 


* 

234.72 


0.53 


Task (T) 


4,649.69 


2 


2,324.85 


6.66 


Pilot- 








Sub j ect 


4,890.94 


11 


444.63 




Interaction 










D X T 


852.03 


2 


426.02 


0.89 


D X S 


4,884.95 


11 


444.09 




T X S 


7,684.98 


22 


349.32 




D X T X S 


10,537.30 


22 


478.97 




TOTAL 


33,734.61 


71 







Table 4. 


MEAN NUMBER 


OF TRIALS TO 


CRITERION 




TASKS A§B 

LEAST 
DIFFICULT 


TASKS C§D 
MODERATELY 
" DIFFICULT 


TASKS E§F 

MOST 
DIFFICULT 


No' Delay 


20.8 


11.0 


11.2 


Delay 


34.2 ' 


9.4 


10.3 


Both 


27.-5 ' 


10.2 


10.7 
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EXPERIMENT 2 

PROCEDURE. All pilot subjects were quite proficient after completion of 

Experiment 1. The flight tasks and operating procedures were familiar to 

them at the beginning of Experiment 2. 

The object of Experiment 2 was to record, for later analysis, each 
pilot siibject's flight control activity while flying assi^ed carrier * 
approaches, with and without the delay condition. Twelve pilots completed 
these tasks. Task B, an easy task. Task D, a moderately difficult task, 
and Task F, a difficult task used in Experiment 1 were chosen for use in 
Experiment 2. 

It was found to be convenient to refer to Tasks D, B and F as Left, 
Center and Right Tasks, respectively, each with Delay (D) and with No-Delay^ 
(ND). Subsequent references to tasks will be made in this manner^ 

•Experiment 2 required each pilot subject to make five successful 
arrestments for each of the Left, Cenl^er, and Right Tasks with and without 
the delay condition. This resulted in a total of 30 successful arrestments 
required of each pilot subject. Successive arrestments were not required. 
Typically, a subject would make 40 to 60 approach attempts in achieving 30 
succesjsful traps. The pilot's control activity was recorded on magnetic 
tape during all ^of his "Approaches, however, only that recorded during suc- 
-^{^ssful approaches, i.e., resulting in arrestment, were subjected to later 
afirialysis. The sequence of tasks flown by each pilot subject was identical. 
The sequence was as follows: 



(1) 


C 


CD) - 


Center with Delay 


(2) 


L 


(ND) - 


Left with No-Delay 


(3) 


R 


(ND) - 


Right with No-Delay 


(4) 


L 


(D) - 


^Left with Delay 


(5) 


R 


(D) - 


Right with Delay 


(6) 


C 


(ND)< - 


Center with No-Delay 



DATA RECORDED. Six (6) pilot control parameters were rec'orded on magnetic 
tape each program cycle. These are: 

DDS - Stabilator Control Stick Deflection 

DSA - Aileqron Control Stick Deflection 

•DRP - Rudder Pedal Deflection 

FSSA - Force Applied to Stabilator Control Stick 

FSAA - Force Applied to Aileron Cpntrol Stick 

FRPA - Force Applied to Rudder Pedal' 
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The three parts of figure 12 are time histories of the six parameters 
recorded during an approach by one of the pilot subjects. These plots are 
typical of all approaches made by all pilot subjects, 

DATA ANALYSIS- VARIANCES IN CONTROL FORCES Afft) DISPLACEMENTS, The first 
step in the analysis of variance was to compute the means and variances of 
each of the six control parameters (DSS, FSSA, DSA, FSAA, DRP, FRPA) 
recorded for each successful carrier approach made during the experiment. 
The results of these calculations were recorded on magnetic tape and were 
listed. 

The next step in the analysis of Experiment 2 data was to average the 
five values of variance for the five successful approaches of a given task 
(Left, Center or Right, Delay' or No-Delay) for each of the six control 
parameters (DSS, FSSA, DSA FRPA), To aid in explaining this and sub- 

sequent steps in the process followed in analyzing the data, consider the 
three dimensional model shown in figure 13. Figure 13 is a sample model 
structure of one of a typical recorded control parameter. Each cell indi- 
cated on the model represents the average variance of the given control 
parameter taken over five successful approaches by one of twelve pilot 
subjects, flying one of three basic approach tasks (Left, Center, Right) 
with one of two visual presentation time delay conditions (Delay or No-DelayJ, 
For example, the upper left-hand cell entry shown on figure 13 represents 
the average of the variances in a variable for five successful approaches 
made by one pilot subject for the left task with delayed visual presentation. 
Table 5 contains the computed average variance values for each of the six 
control variables (DSS, FSSA, FRPA) for each pilot subject (12 pilots) 

for each task (Left, Center, and Right) for the two delay conditions (Delay 
or No-Delay), 

At this. point, it is important to draw attention to what may be 
subtle enough to confuse. Note that the analysis discussed in the remainder 
of this section is an analysis of variance in variances. 

An Average of Statistics program, figure 14 (4 parts), calculated an 
average variance for each cell of the model. The' same program was used' 
to compute the average variance of the variances of each control parameter 
for all pilot subjects in each of the three tasks (Left, Center, Right) 
with and without delay. The results are summarized in figures 15, 16, arid 



Since the entries in cells are the average variance in the control 
paTameters for a specific pilot and flight condition, the differences in 
these entries represent the effect of the flight conditions on the manner 
in which pilots exercise their piloting skills. 

The results of averaging the cell entries, shown in figure 13, over 
all pilot subjects are given in tables 6, 7, and 8, and are plotted in 
figures 15, 16, and 17. Notice that for all three starting positions the 
delayed visual task had greater variance than the non-delayed for the 
following parameters: longitudinal control deflection (DSS), lateral 
control deflection (DSA) and lateral control force (FSAA). In fact, in 
only four of the eighteen comparisons of variance (Left, Center and Right 
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Figure 12. Sample Time History (Longitudinal) 
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Figure 12. Sample Time History (Lateral>-(CQNT) 
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Longitudinal Contrpl Deflection (DSS) 




EXAMPLE 

This cell contains the 
average variance 
five successful 
approaches by 
pilot PI 
flying „the 
left task 
with delay 



D 
ND 



12 Pilot Subjects 
LEGEND 



D, ND 
L, C, R 



P. 

1 



Delay Condition 
Task Origin — 
Left, Center, Right 
The Pilot Subjects 



Figure 13. Analysis of Variance Model Sample, 
Experiment Part 2 , ; 
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FILBT SLEwECT AVfrRAGE VARIANCE 
BY TASK, CELAY ANC 

C0NTfif?L PARAMETERS 



FiLQT 


CELAY ^R /VEf^A 
se DELAV LEFT 


G£ VARIANCE 
CFNTFR 


RIGHT 


PARAMETER 


aaaa 

AA^A 


r 

NC 


.733?! 


•7^278 


1 


•02865 ' 
•6S76S 


DSS 
DSS 




C 

NC 


1.3BTa2 
1.3?A^8 


•6151C 
1 • 34SC6 


I, 
1 


•4C882 


DSS 


EEEE 
EEEE 


C 


?.Ccl2S 
1.75S55 


•369lfc 
• 74C5« 


3 
2 


.4C512 
.C7053 


DSS 
DSS 


icic 
icic 


c 

KC 


*■ 


•6333i: 
.7fc{'£9 


1 


.5*1826 


DSS 
DSS ■' 




c 

NC 


.779tfC 


1 MSP^l 
•71521 


, 1 


•^1636 
• r ' ^ b 


DSS 


-3333 

3^33 


c 


1.1213^ 
1 .C^^S? 


.7975? 
. 45345 


1 
1 


•6C45C 
• n979r 


DSS 
DSS 

i>/ w 


2121 
2121 


c 

NC 


J .3^445 
1,3678C 


1 •C1772 
1 . 18«2t=5 


1 

2 


•^6072 

• 39^9 Ji* 


DSS 

D^S 

lb? w 


3C3C 


C 

NC 




. S759«^ 


1 

1 


•41^65 

* A A ^ ix 


DSS 

»^ C" 0 


^^^^ 
^^^^ 


C 

\c , 


1.56314 


3»lRC2 3 
.5C685 


2 
? 


•C7885 ' 
•4S766 


CSS 
DSS . 


w ^ •w' 

5655 




c. H'=-»4 

1 A C 1 /■ 1 

I • c 2 1 1 1 


.^2Cf t. 

. 0 H V, u 


1 


• S7;^41 

• i VJ t ^ •f 


DSS 


6666 


c 


1 .364?C 
1.27f 45 


2»197»3 
2 .^=8591 


2 
2 


•692'»S 


DSS 
DSS 


SS9S 

ssss 


c 

NC 


,6^a?4 
1 .lfc'^78 


.76724 
1 •5663d 




.e8C8 6 
.5CP32 


DSS 

DSS 


aaaa 
aaaa 


n 

NC 


.SP7S7 • 
.75771 


.68062 
•9^036 


1 
#^ 
c 


.91662 
.0382-7 


FSSA 
FSSA 




c 

NC 


.•.6t77'^ 
• .6^?72 


»465C? 
.63657 


1 


.9865ft 
.423C1 


FSSA 
FSSA 


EEEE 
EEEE 


C 

^C 


l.CS6?5 
1 .31r4« 


.35884 
•5yi7fr 


2 


.15772 
.331C3 


FSSA 
FSSA 
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.FILST SUByEtTv AVERAGE VARIANCE 

Ev /Task, ceuay and . 
CQ'KTRei. Parameters 



PlL8T CELAV 9R AVERAGE VARIANCE 
CQDe- N8 delay left CENTER F 



Right 



P-ARAMETER 



icic 


C 




.62122 




.51924 


1^38129 


FSSA 


iClr 


IN U 








• HO J 




r SoA 




D 






1 


•49C83 


1^38756 


■ FSSA 








• / I / r / 




• wOc4 1 


3.C4751 


FSSA 




c 




•6PS51 




•34999 


.77852 


FSSA 


2l2l 






• ^4A^r 




■ c / c o *t 


• O 1 A 1 fi 

.y loio ^ 


rcc A 
r ooM 


3C3C 


C . 




.82738 




.7174P 


i. 77233 


FSSA . 


2C3C 


NC 




.81291 




.46073 


1.87121 


FSSA 


3333 


C 




.?8782 




.6671G 


1.34463 


FSSA 


Jo 


NL 








. 35S3Q 


1 .02932 


FSSA 




c 




.93118 


2 


.C1811 


1.39633 


FSSA 


■ y 


NC 


1 


.64432 




.44698 


2.26562 


PSSA 


w *^ w J 


C 


1 


.16170 


/ 


.66272 


1^0.65?4g 


FSSA 


w 3 3 






Q it Q Q 




* C O ✓ J. 

•co764 


l^l983l 


r* r« A 

FSSA 


6666 


c 


1 


»C7^44 


1 


•36543 


2»r9842 


FSSA 


6666 


IN w 




» ' *^ 1 ^ 




?7c -at; 

• C ' -1 


C ii 6 A 7 


COCA 

r boA 


S599 


C 




.59769 




•76C84 


i»i9i6g 


FSSA 


9599 


NC 




% f^f^O 1 1 
1 c c c X 1 






- c A o ii 


rcQ A 


^A AAA 


a 

D 


1 






.3t>394 


l^S39a3 ^ 


DSA 


A*AA 




f 


1 35853 




» X ^ ~ *T " 


1 • 4A i P 


CSA 




c 


\ 


» w A 




%l:55ri 


A • / ' w C 


OSA 




\C 


1< 


.46r45 




•08837 


1^24926 


DSA 


^eeee 


C 


1. 


.34147 




•25545 


2^C8146 


CSA 


eeee 


NC 


1. 


.32301 




•C94C9 


1»4015£ 


DSA 


icic 


C 


2 < 






•08501 


3^40622 


DSA 


ICIC 


NC 


1< 


.60034 




•321CU 


^•512C6' 


nSA 


2C2C 


C 


1< 


.64534 




•325S6 


1»5486'7 


DSA 


2C2C 


NC 




,87779 




•17638 


W9266 


DSA 


2121 


c- 


1. 


.47feS4 




•06181 


•79037 


DSA 


2121 


NC 


i. 


•13744 




•C77C5 


•59372 


DSA 
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f TABLE'S (CeM) 

FILST SuBiECT AVERAGE VARIANCE 
PY TASK^ DELAY AND 
. CeNTReL PARAMETERS 



PIL8T 


ceuay 


. AVERAGE VARIANCE' 


PARAMETER 




Nfi 


ORLAY 


LEFT 


CENTER 


RIGHT 




3C3C 


u 


>% , 








DSA 


C V w 


NT. 






•C76C5 


•9974C 


DSA 

W 


3333 


.D 


2 • 


96739 


•97257 


4*36092 


DSA 


3333 


NC 




63645 


'•10219 


2*60337 


DSA 




u 










\DSA 


1 ' ' T 


NC 






•10321 


•93032 


DSA 

W \J 


5555 


C 




C C 7i 


• 4oo9 1 




DSA 


5555 


NC 






•35583 


^•24545- 


DSA 


6^66 


p 

u 


1 • 


^ /Ic 1 


A C C\ /1 0 4 


1 • 00 Job 


DSA 




■NC 


1. 


24?65 


•5651S 




DSA 


S595 


r 


J. • 


C f z ^ / 


• C w J 0 U 


1 • i c / 0 / 


DSA 


5 > ^ ^ 


NC 




7793j' 


•2^390 


•73552 


D&A 










"A 




aaaa 


pi 

u 






• 0 PC C 




FSAA 


A AAA 


NC 




1U7'5 


•C1565 


•11719 


' FSAA 


c-cc 


U 






• ^ 1 OG h 

• C i C ^ *f 




FSAA 


CCDC 


NC 




C5123 


•CC366 


•060G2 


FSAA 


E££E 


r> 

U 










FSAA 


u ^- 1» 


NC 




C544C 


•C0329 


•0817C ■ 


FSAA 


icic 


c 




Q •! Z. 




• logOl 


FSAA 




NC 




C8C43 


•C2l68 


•13331 




Z02C 


D 




C6^35 


•CUlC 


• 06051 - 


FSAA 


cC2C 


NC 




C3337 


' •GC669 


•09268 


FSAA 


2121 


c • 




C7135 


•00498 


• 0'^314 


FSAA 


2121 


NP 




C6586 


•C0313 


. ^03407 


FSAA 


303C 


C 




'C19CC 


•00560 


*C'f682 


FSAA 


3C3C ■ 


NC 




CUSC 


•C0338 


' •C4652 


FSAA 


3333 


D 




1373S 




■ ^20388 


FSAA 


3333 


NC 




12H4 


.C1173 


•1271C 


(;SAA 




C 




C2514 


•GC899 


•04942 


FSAA 








C2625 


•00374 


' •0^^77. 


FSAA 
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TABLE 5 '(CesT) 

PILST StB».ECT AVERAGE VARIANCE 
PV TASK, DELAY ANO 

C9M^6L Parameters • 
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fIl8T cei^av or average Variance 
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PARAMETER 



\ C9De 


N9 


DELAY LEFT 


CE^sTeP 


R-I6HT 






\S - 55 


D 




•C3G65 
•C2252 


•15858 
•12288 




FSAA 
FSAA 


6666 
• 6666 


c 

NC 


.05565 
.05698 > 


• casi'i'"-''' 

^-~«jQ38g1 
•jC17S6 


•10222 
•11338 




FSAA 
FSAA 


SS9S 
eggs 


C 

NC 


..C7?1C 
.C53li£ 


•C773i 
•C6365 




FSAA 
FSAA 


AAAA 
A^AA 


D 

NC 


.CC^6'6 
.CC271 


•CC132 
♦CC1C7 


»CC327 
•0C338 




DRP 
ORP 


^ CCDD 
CCDC 


D 

NC 


.CCC3C 
.CCOC 


*C0C/f7 
•C0C3C 


•^CCC4E 
•00040 




DRP 
DRP 


EEEE 
EEEE 


C 

KE 


.00732 
.C1U8 


•CC263 
•C0C49 


• cicc^ 

•ClC80 




DRP 
DRP 


icic 

IClG 


C 

NC 


.0538'* 
.04678 


•coisi 

•C0643 


•^6762 
•C4623 




.DRP 
DRP 


2CcC 


NC 




•OClcO 
•CC163 


•05038 
•01839 




DRP 
, DPP 


clSl 


r. 

NC 


.^♦752 
.07PCS 


^■•C01c3 • 
•CC217 


•01249 
•02113 




DRP 
DRP 


3030 


c 

NC 




•C0639 
•00377 


• 03259' ■• 

• Cf4l26 




DRP 
DRP 


3333- 

3333 


• c 


.coiei 

.00057 


•CC021 

'-^ClOl 


•00157 
•01463 




DRP 
DRP 




NC 


.00^79 
.0C'3O 


•C"DC.82 
icOC51 


•00558 
•01207 




DRP 
DRP 


K55S 


C 

NC 


.00161 

• 


•C0035 
•C007C 


•00128 
•00173 




'drp 

DRP 


6666 
6666 


C 

NC 


.043^4 . 

.CS'^'lC 


•00190 
•C2437 


•05229 
•C451C 




DRP 
DRP 


sS9g 
ssgg 


C 

NC 


.C7378 , 
.03750 


•C027C 
•C1150 


• 025U 
•03298 




DRP . 
DRP 
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TABLE 5 (C9NT) 



\ 



F-^t^ si^mECT average Variance 

eV DELAY AND 

CONTROL PAR^f<-ETERS 



FlLST 
C9CE 

AAAA 
A*AA 

cccc 

EEEE 
EEEE 

ICIC 
ICIC 

2C^ 
2:2c 




3333 
3333 

5555 
5555 

6666 
6666 

S5SS 
S59S 



CFLAV ^fi ^/iVEf^AGE VARI^CE. 

K8 DELAY LFFT CENTER 'RlQhT 



C 

ND 
r 

•ND 
C 

ND 



1.C535? 




.S2^38l\.3 
.31726 ^ 



l.A15^5 •76C66; 



KD 


2.7157'^ 


l.>7^ 




u!t^4 


FRPA 


C 


3.Z73':A 


• -"^3384 


4i 


^59249 


\ ^ FRPA 


ND 


1 .7S5cS 


• 861it4 


■41 


'21057 




C 


3.81)5!?C 


•9325S 


5. 


•416C6 


^RPA 


ND 




.93558 


12' 


'3671^ - 


FRPA 


C 


T""~>^S4^65 


5*2042^* 


18' 


>69?86 


FRPA 


ND 




2*56332 


3C' 


•16136 


FRPA 


C 


1.53-'!7*t 






•'26641 


FRPA 


ND 


..471 '♦S 


•73860 




•5C758 


FRPA 


C 


1.45A13 


•4664n 


1< 


•756C7 


FRPA 


ND 




•.'•«263 


4( 


•77768 ' 


FRPA 


C 




•2441^ 




• 7^'^23 


FRPA 


NC 


..4Ca73 


•292S6 




.4775c 


> FRPA 


c 


a. 00*^^3 


1 ^4367^ 


19 


'1781S 


FRPA 


ND 


.Slf 54 


12»327SC 


11, 


.41264 


FRPA 






1 •432C5 


11 


.75544 


FRPA 


ND 


'5,75441 


7 1 66 ISC 


12 


.g026.C 


FRPA 



7779c 
9897S 

0^083 
50718 



747^3 



PARAMETER 

FRPA 
FRPA 



FRPA 
FRPA 



FRPA 
FRPA 

FRPA . 
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Statistics Program 
Calculates means yCA. and vari- 
ances {6^ of 5 parameters (DSS, 
FSSA, pSA, FSAA, DRP, FRPA) for 
each pilot, each carri en approach 
tri al . \ 




i 



Average of Statistics 

Calculates the averageyuand ^^for eacK pilot for their 5 
carrier approaches for each task for each' parameter (DSS, FSSA, 
pSA, FSAA^ DRP, FRPA). Collects average mean and variance 

I <y ^>^s ) 'for each task and delay condition for each para- 
meter (DSS, FSSA, DSA, FSAA, DRP, FRPA). 
LEFT (D,ND), CENTER (D,ND), RIGHT (D,ND) 




Figure 14. Flow Diagram of Experiment Part i 
Data Processing 

42 44 




ANOVAS Program . 

^Analysis of variance of variances in 
parameters (DSS, FSSA, DSA, FSAA, DRP, 
FRPA) due to delay/no delay for all 
subjects for each separate task (left, 
center, and right). 

A X S Analysis of variance 
A Delay or no Delay 
S Pilot Subjects 



TASK 
D ND 
A X S 
LIST 



ANOVABS Program 

Analysis of variance of variances in 
parameters (DSS, FSSA, DSA, FSAA, DRP, 
FRPA) due to dela^y/no delay and due to 
task (left, center, right) dTFFerences 
for all pilot subjects. 

A X B X S Analysis of variance 
A Left, Center, Right'task 

B Delay or No Delay 

S Pilot Subjects 



A X B X S 
D ND 
LIST 



Figure 14. Flow Diagram of Experiment Part 2 
Data Processing CCONT) - 
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AVERAGES 

Compute averages of control parameters 
and subtract from time histories to 
form modified time histories. 



I 



AUGMENT 

Augment modified time histories out to 
1024 data points for all runs with 
zeros for DFFT processing. 



DFFT 

Compute DFFT's for all 
histories. 



modified time 
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AVERAGE 

Average DFFT's for each harmonic 
for each subject over his 5 runs 
for a particular task. 




Verifica- 
tion 
listing 



LIST 

Mean DFFT 
for givpn 
task ,for all 
pilots 



AVERAGE -1 

Average and compare DFFT's for 
each harmonic for all pilot sub- 
jects for a particular task. 



CALCOMP 
Mean DFFT 
for given 
.task for all 
pilots 



LIST 

Mean DFFT 
for all 
tasks for all 
pilots 



AVERAGE -2 

Average and compare DFFT's for 
each harmonic for- all subjects for 
all tasks. 



CALCOMP 
Mean DFFT^ 
for' all 
tasks for 
all pilots 



figure 14. Flow Diagram of Experiment Part 2 
Data Processing (CONT) 



ERIC 
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Table 6. STATISTICAL SUMMARIES LONGITUDINAL 
AVERAGE VARIANCES-! 2 SUBJECTS 



DISPLACEMENT (DSS) 





LEFT 


CENTER 


RIGHT 


ROW 
MEANS 


DELAY 


1.4068 


1.1192 


1.6906 


1.4055 


NO 

DELAY 


1.2698 


1 .0398 


1.6401 


1.3166 


COLUMN 
MEANS 


1.3383 


1 .0795 


1.6654 





V 



Task = 
'Task = 
"Delay = 

Delay 

"Task X Delay = 
'Task X Delay = 



6.2534** 
.0072 
.8934 
.3674 
.0428 
.9585 



FORCE (FSSA) 





LEFT 


CENTER 


RIGHT 


ROW 
MEANS 


DELAY 


.8546 


.8666 


1.5983 


1.1065 


^ 

NO 

DELAY 


.8395 


.6460 


1.7487 


1.0781 


COLUMN 
MEANS 


.8471 


.7563 


1.6735 





Task 
Task 
"Delay 
'oelay 

"Task X Delay 
'Task X Delay 



39.. 7728** 
.0000 
.1210 
.7332 

1.0796 
.3582 
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* STATISTICALLY SIGNIFICANT AT .05 LEVEL 
** STATISTICALLY SIGNIFICANT AT .01 LEVEL 
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Table 7. STATISTICAL SUMMARIES LATERAL AVERAGE 
VARIANCES-! 2 SUBJECTS 



DISPLACEMENT (DSA) 





LEFT 


CENTER 


RIGHT 


ROW . 
MEANS 


DELAY 


1.6824 


.3232 


1.9945 


1.3334 


NO 

DELAY 


1.4481 


.2015 


1.5918 


1,0804 

• 


COLUMN 
MEANS 


1.5652 


.2623 


1.7932 





Task = 40.5225** 

'Task = ■ .0000 

'Delay = 10.2748** 

'Delay = .0083 

'Task X Delay = 1.8443 

'Task X Delay = .1804 



FORCE (FSAA) 





LEFT 


CENTER 


RIGHT 


ROW 
MEANS 


DELAY ^ 


.0738 


.0187 


.0988 


.0637 


NO 

DELAY 


.0660 


.0126 


.0865 


.0550 


COLUMN 
MEANS 


.0699 


.0156 


.0926 





Task 
''Task 
^Delay 
^Delay 

^Task-X Delay = 
'lask X Delay = 



42.6083** 
.0000^ 
5.3550* 
.0392 
.4156 
.6701 



* STATISTICALLY SIGNIFICANT AT ..05 LEVEL 
** STATISTICALLY SIGNIFICANT AT .01 LEVEL 
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Table 8. STATISTICAL SUMMARIES DIRECTIONAL 
AVERAGE VARIANCES-! SUBJECTS 



DISPLACEMENT (DRP) 





LEFT 


CENTER 


RIGHT 


ROW 
MEANS 


DELAY 


.0248 


.0017 


.0219 


.0161 


NO 

DELAY 


.0197 


.0045 


.0207 


.0150 


COLUMN 
MEANS 


.0222 


.0031 


.0213 






FORCE (FRPA) 






LEFT 


CENTER 


RIGHT 


ROW 
MEANS 


DELAY 


4.5721 


1.1846 


6.5518 


4.1028 


NO 

DELAY 


3.3319 


2.5284 


7.6548 


4.5055 


COLUMN 
MEANS 


3.9520 


1 .8565 


7.1033 





Task = 8,4328** 

p 

Task .0022 

''Delay' = .3393 

^Delay' = .5775 

''Task X Delay = 1.2694 

''Task X Delay = .3007 



'^Task 


8.0062** 


P 

^ Task 


.0028 


''Delay 


.5977 


''Delay 


.4613 


''Task X Delay = 


1 .0052 


'^Task X Delay = 


, .3839 
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* STATISTICALLY SIGNIFICANT AT .05 LEVEL 
** STATISTICALLY SIGNIFICANT AT .01 LEVEL 
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Figure 15. Comparison of Variances - Longitudinal 
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Figure 16. Comparison of Variances - Lateral 
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Figjjjsre 17. Comparison of Variances - Directional 

31 53 



NAVTRAEQUIPCEN IH-250' ' , : 

• * 

Tasks for six control parameters) did the non-delayed variance exceed the 
delayed variance. Of these, three were for the rudder control variables 
(Center Task, rudder pedal deflection (DRP) and Center and Right Tasks, 
nidder control force (FRPA)). 

There is some chance that the differences in average variance observed 
between the different conditions are due to chance or happenstance. One < 
accepted method for determining the probability that the observed phenomena 
are due to chance is known as the multivariate analysis of variance. There- 
fore, a multivariate analysis of variance in the average variances of the. 
six control parameters was performed to determine statistical significance 
of the differences in average variance of the control parameters due to 
both the task and the delay factors. For our purposes, two levels of 
statistical significance arg considered and are defined to be those situa- 
tions in which the F ratios resulting from delay effects being due to 
chanc^, are less then .05 or less than .01. 

The analysis of variance' program was obtained from what is known as 
VUL2, the Vanderbilt Statistical Package , written by Dr. Laird W. Heal. 
The program, cal led. ANOVABS (figure 14, part 2), calculates-an analysis of 
variance with two "within" factors, or repeated measures. 

The differences in variances of the control inputs (DSS, FSSA, DSA, 
FSAA, DRP, FRPA) for the two basic conditions of Delayed and Non-Delayed 
visual presentation are shown in figures 15, 16, and 17. The results of 
the multivariate analysis on the differences are presented in tables 6, 7, 
and 8. The F ratios for the task origins are statistically significant for 
all tasks. This indicates that all of the observed control parameters were 
exercised differently for each task. This is not surprising since the 
tasks are all different. The center task required the fewest control mani- 
pulations of the three tasks. The principal difference in left and right 
task was the addition of the turbulent air variable to the right task. The 
F ratio based on the differences of variances due tx> delayed or non-delayed 
visual presentation for the lateral control parameter is statistically 
significant at P = .0083 for the lateral control deflectiofi and at P = .0392 
for the lateral control force. 

FOURIER ANALYSIS OF CONTROL INPUTS, The question "If pilots do perform their 
skills differently when visual stimuli are delayed 100 ms, in what^ way(s) 
is their performance different?" is difficult to answer by examining the 
time histories of the pilot »s control activity. One time history appears 



Mendenhall, William, "Introduction to Probability and Statistics", Third 
Edition, Duxbury Press, pp. 243f. 

7 

Heal, Laird W., "VUL2 Vanderbilt Statistical Package", Xerox Computer 
Users* Group Exchange Program Library, Catalog No. 890400-1 IBOG. 
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much the s%me as another and in particular, the time histories for'^the" 
Delayed and Non-Delayed cases also appear to be very similar. The results 
of the multivariate! analysis of variance in Control Forces ^d Displacements, 
"indicate that clea^^dj^erences exist in the variances of the varioxxs control 
,parameters, but not^^^rti;^ the nature of those differences might be. One -method * 
^of ex^ining time histories is to investigate their frequency content. The 
tim^rfptorifes were mappe?ff into- the frequency, domain to better evjiluate the 
exa^pfc.ture of the differences ^which occur in piloting technique when the 
pilot subject *s visuatl stimuli have. been delayec^ The Fourier transformation 
to the frequency domain was accomplished by using a published program package* 

The Discrete Fast Fourier Transform (DFFT) is one convenient tool £tft 
performing the required iftapping from the time domain into the frequency do- 
main. One computer program, FOURT, processes the Cooley-Tukey Fast Fourier 
Transform as defined by: 



N-1 -i2 7r mn 

^ m=o m 



0 < n < N-1 



iVhere: i = imaginary 

summation index on the number of data points 
harmonic 

V 




N 



number^of^data points in the recorded time history 



» m th value of \the untransformed data 
3^ \ = . amplitude of tl^ n th harmonic of the transform 

An error analysis of this program appears in a related publication.^ 
The various time histories were of differing length making comparisons 
of the results of the Fourier^j)rocessing difficult. The different lengths 
were all augmented with zeros to make ^eir lengths 1024 data points, 
(figure 14, page 44, Augment) allowing faster program execution and a common- 
ality of fundamental frequencies of the Discrete FouV^er Transform (DFT) . 
So as not to introduce major harmonic. content into the DFT's, the average 
value of each time history was removed bef^e augmentation of the data 
strings. 



g 

Brennpr, M., "Three Fortran Programs that Perform the Cooley-Tukey 
Fourier Transformation,** MIT Lincoln Laboratory Publication AD 657019, 
28 July 1967. • * 

9 

Ferris, James F. , and Nuttall, Albert H., "Comparison of Four Fast Fourier 
Transfoim Algorithms, "-NUSC Report No. 4113, 3 June 1971, Naval Underwater 



'^>^3*ems Cen^r, Newport, t^.I. 
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TTie ch^^ sampling period results in a 

'tunciamentar"5requency of .0195 Hz per frequency cell. Since preliminary 
analysis of selected time histories of each control parameter representing 
each task indicated no appreciabTe energy in the spectra at frequencies 
above 4 Hz, the calculations were halted .at 200 harmonic?. 

The following spectra were calculated for each control parameter. 

i 

a. » For each pilot subject, the spectra for five successful approaches 
for each task iti each delay condition were averaged. (Figure 14, page 4^, 
Average 1.) This produced three (one for each task) spectra for each delay 
condition for each pilot subject. 

b. Each of the six spectra thus produced per pilot subject were then 
averaged over alTthe pilot subjects, providing six spectra for the entire 
group of pilot subjects, one for each task for each delay condition (figure 
14> page 45, Average 2). 

Thus,, thirty-six spectra were prepared for the entire groiip of pilot 
subjects; three (Tasks) x two (Delay Conditions) x six (Control Input 
Parameters). These are presented in figure 18. Figure 18 also displays 
the differences in the spectra discussed above, i.e., the differences in 
the spectra for the Delayed visual presentation condition minus the Non- 
Delayed visual presentation condition. These differences were computed 
by subtracting the real and imaginary amplitudes, for each frequency cell 
of the delayed spectrum from the real and imaginary amplitudes of the 
same frequency' cell of the Non-Delayed spectrum. Figure 18 shows the 
general trend of the results of pilot activity in the frequency domain. 
Notice that the control input spectra have decreasing amplitude with 
increasing frequency and that the difference spectra (Delay spectra minus 
the No-Delay spectra) have the same general trend. This suggests that 
the delay effects (as indicated by the difference .spectra) are functions 
of frequency and that the effects are greater at around .6 Hz. The results 
of the frequency analysis are summarized in table 9. The principle fre- 
quency and approximate amplitude refer to the difference spectra of the 
delayed condition minus the n'on-delayed condition. The control input 
limit refers to either the delayed or non-delayed case and merely states 
the approximate upper frequency limit of information content. 



10 i> --^ 

The spectrum averaging discussed herein is the arithmetic mean of the 

contents of each frequency c\^l (multiple of the fundamental frequency). 
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Figure 18. Fast Fourier Transforms of Control Inputs 
by Task 5 '7 
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Figure 18. Fast Fourier Transforms of Control InputV 
by Task (CONT) N 
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Figure 18. Fast Fourier Transforms of Control Inputs 
by Task (CONT) 
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Figure 18. Fast Fourier Transforms of Control Inputs 
by Task (CONT) 
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Figure 18. Fast Fourier Transforms of Control Inputs 
'""^ by Task (COOT) ^ . 
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Figure 18. Fast Fourier Transforms of Control Inputs 
by Task XCONT) 
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Figure 1^. Fast Fourier Transforms of Control Inputs 
by Task (CONT) 
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Figure 18. Fast Fourier Transforms of Control Inputs 
by Task (CONT) 
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Figure 18. Fast^-E^urier Transforms of Control Inputs 
by Task (CONT) ^, - 
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Figure 18. Fast Fourier Transforms of Control Inputs 
by Task (CONT) 
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Figure *18. Fast Fourier Transforms of Control Inputs... 
by Task (CONT) 
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Figure 18^""f^ast Fourier ^ansforms of Control Inputs 
by Task (CONT) 
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Fieure^lS. Fast Fourier ^ransfoims of Control Inputs 
^ ^ by Task (CONT) 
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Figure 18. ' Fast Fourier Transforms of Control Inputs 
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Figure 18. Fa§t-^urier Transforms of Control Inputs 
by Taslc-f€Q^^^l _^ ^ 
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Figure 18. Fast Fourier Transforms of Control IntHits 
, by Task (CONT) 
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Figure 18. Fast Fourier Transf oims*tJf*<Jpntrol Inputs 
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Figure 18. Fast Fourier Transforms of Control Inputs 
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Table 9. SUMMARY OF ANALYSIS OF FAST FOURIER PROCESSING 



CONTROL 
PARAMETER 


LEFT 


TASK 
CENTER 


RIGHT 


ncc 
uoo 

Principle Frequencies (Ha) 
Approx. Amplitudes 
Control Input Limits (Hs) 


.2f^o .4 
.08 ' 
.8 


.1 

.16 

.8 


.1, .4 
.07 
.8 


rboA 

Principle Frequencies (Ha) 
Approx. Amplitudes 
Control Input Limits (Hs) 


.4 

.05 
1.6 


.2 to 1, Peak .6 
.04 
2.4 


.4 

J .-OS 
2.4 ^ 


UoA 

Principle Frequencies (Hs) 
Approx. Amplitudes 
Control Input Limits (Hs) 


.1 
.1 
.8 


.1 to .6 
.025 
1.2 


.2 
.1 
.8 


FSAA 

Principle Frequencies (Ha) 
Approx. Amplitudes 
Control Input Limits (Ha) 


.1 

.02 

.8 


.2, .4 

.006 

.8 . 


.2 

.02 

.8 


Ul\r 

Principle Frequencies (Ha) 
Approx. Amplitudes 
Control Input Limits (Ha) 


.1 

.015 
1.6 


.1 

.006 
1.6 


.1 to .4 
.006 
1.6 

0 


FRPA - 

Principle Frequencies (Ha) 
Approx. Amplitudes , 
Control Input Limits (Ha) 


.4 
.1 
2.4 


.4, 1.2, 1.8 
.04 
3.4 


.4, .9 
.12 •. 
3.4 
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SECTION IV 
CONCLUSIONS AND DISCUSSION 



The first question posed in the statement of the problem **Does 100 ms 
delay of a visual presentation affect pilot learning performance?" was 
answered by Experiment 1. No statistically significant differences were 
found between the "trials-to-criterion" (three successive traps) in the 
Delayed condition and in the Non-Delayed condition. 

The second question posed in the'statement of the problem **Do pilots 
perform their piloting skills differently when their visual stimuli have 
been delayed for 100 ms?*' has been answered in the affirmative insofar as 
the piLot control inputs in the lateral control parameters (displacement 
and force) are concerned. The effect of delay was found to be statistically 
significant at the .0083 level for aileron control displacement (DSA) and 
at' the .0392 level for aileron control force (FSAA). The effect of delay 
on the remaining four control parameters (DSS, FSSA, DRP, FRPA) was foiind 
to be not statistically significant. 

While the differences in the mean scores for all tasks for the remain- 
ing four pilot* control input parametfers for the Delay compared with the No- 
Delay condition were all statistically not significant, it is interesting 
to note that of the eighteen mean comparisons made (see tables 6, 7^ and 8), 
four average variance values were less for the Delay condition than for the 
No-Delay condition. (They were elevator control farce (FSSA) during the 
Right Task, rudder pedal deflection (DRP) during the Center 'i;ask, and rudder 
pedal force (FRPA) during the Right and Center Tasks.) 

It is believed that these four average variance values can be explained. 
Three of the four comparisons involved rudder control force and/or deflec- 
tion. Several approaches by subject pilots were ma<le with high angle of 
attack, sufficient to activate the rudder pedal stall warning shaker. It 
is believed that the directional displacements and forces recorded due to 
the shaker masked the effect of the delay condition on pilot subject induced 
control displacements and forces. The fourth comparison, elevator control 
force during the Right Task, is believed to be similarly masked by the 
rough air turbulence used in this task. None of the other tasks utilized 
rough air turbulence. 

The third question, "If pilots do perform their skills differently 
when visual stimuli are delayed 100 ms, in what way is their performance 
different?" has been resolved by transforming the pilot control inputs to 
the frequency domain and comparing the frequency spectra gf the control 
inputs for the delayed visual 'presentation to the spectra for the non- 
delayed visual presentation case. These comparisons are summarized in 
table 9. 

The time histories of-each control parameter for all successful 
approaches were transform^ed to the frequency domain using the discrete 
Fourier transform. T\\e transformations were averaged for each given task 
and each delay condition over all pilot subjects. The difference spectra 
were formed by subtracting the average delayed spectrum from the average 
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non-delayed spectrum for each task and each control parameter, (figure 18). 
The difference spectra show the effect of delay to^ecrease with |,ncreasing 
frequency. The major difference between the Delayed and No-Delayed spectra 
typically occurred in the range 0 to 2 Hz^ 

The results of these experiments are applicable to a high performance 
simulation (F-4) using a narrow field of view visual presentation. However, 
caution should be exercised before any attempt is made to extrapolate the 
results to visual systems with wider fields of view or to aircraft having 
different frequency modes such as large bomber or transport aircraft. 

_ In conclusion, it has been determined that learning performance of 
pilot subjects, executing the tasks specified for Experiment 1 and in the 
simulator system utilized, was not affected by 100 ms delay in visual 
stimuli. Perhaps this result could be due to pilot subjects responding, 
with extra effort, to the delayed task conditions, i.e., they may have 
"tried harder.** It was determined that, in general, pilot subjects 
manipulated their flight controls differently both in displacements and 
in control force when their visual stimuli were delayed 100 ms. These 
differences are indicated both by the general trend toward a greater 
variance in control activity (in some cases the differences were statis- 
tically significant} and by the differences in the frequency spectra for 
the Delayed and Non-Delayed conditions. 
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SECTION V 
RECOMMENDATIONS 



r 



In view of the findings of Exp'eriments I and 2, the following studies 
are recommended: 

a. A Similar study 'be conducted which would allow both a variable 
time delay and variable task as independent arguments. Sample areas of 
interest would include: learning performance and input control variance 
a5 functions of length of delay time and task type. Since the present 
experiments considered only the carrier landing task, other task types 
might be aerial ^^ueling, air-to-ground weapon delivery, and formation 



flying. 

b. A similar sttfd^ should be conducted for a large field of view 
visual presentation system. 



c. 



i^uc 



A similsr study should be conoucted for large muliti-engine 
transport type aircraft whose natural frequencies are vastly different 
than the strike type of aircraft (the P-4) used in these studies. 

d. The study should be repeated utilizing predictive filters 
designed based upon the frequency^ spectra of the differences in the 
delayed and the non-delayed pilot control input performance. The pre- 
diction could be expected to reduce the effects of the delayed visual^ 
presentation. . ' • ' 
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